Methods, apparatus and system for rendering an audio program

ABSTRACT

A method for generating a bitstream indicative of an object based audio program is described. The bitstream comprises a sequence of containers. A first container of the sequence of containers comprises a plurality of substream entities for a plurality of substreams of the object based audio program and a presentation section. The method comprises determining a set of object channels. The method further comprises providing a set of object related metadata for the set of object channels. In addition, the method comprises inserting a first set of object channel frames and a first set of object related metadata frames into a respective set of substream entities of the first container. Furthermore, the method comprises inserting presentation data into the presentation section.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional and claims the benefit of priority U.S.patent application Ser. No. 15/516,509 filed Apr. 3, 2017, which claimspriority to International Patent Application No. PCT/EP2015/072667 filedOct. 1, 2015, which claims priority to U.S. Provisional PatentApplication No. 62/146,468 filed on Apr. 13, 2015 and European PatentApplication No. 14187631.8 filed on Oct. 3, 2014, all of which arehereby incorporated by reference in their entirety.

TECHNICAL FIELD

The present document relates to audio signal processing, and moreparticularly, to encoding, decoding, and interactive rendering of audiodata bitstreams which include audio content, and metadata which supportsinteractive rendering of the audio content.

BACKGROUND

Audio encoding and decoding systems enabling personalized audioexperiences typically need to carry all audio object channels and/oraudio speaker channels that are potentially needed for a personalizedaudio experience. In particular, the audio data/metadata is typicallysuch that parts which are not required for a personalized audio programcannot be easily removed from a bitstream containing such personalizedaudio program.

Typically, the entire data (audio data and metadata) for an audioprogram is stored jointly within a bitstream. A receiver/decoder needsto parse at least the complete metadata to understand which parts (e.g.which speaker channels and/or which object channels) of the bitstreamare required for a personalized audio program. In addition, strippingoff of parts of the bitstream which are not required for thepersonalized audio program is typically not possible without significantcomputational effort. In particular, it may be required that parts of abitstream which are not required for a given playback scenario/for agiven personalized audio program need to be decoded. It may then berequired to mute these parts of the bitstream during playback in orderto generate the personalized audio program. Furthermore, it may not bepossible to efficiently generate a sub-bitstream from a bitstream,wherein the sub-bitstream only comprises the data required for thepersonalized audio program.

The present document addresses the technical problem of providing abitstream for an audio program, which enables a decoder of the bitstreamto derive a personalized audio program from the bitstream in a resourceefficient manner.

SUMMARY

According to an aspect a method for generating a bitstream which isindicative of an object based audio program is described. The bitstreamcomprises a sequence of containers for a corresponding sequence of audioprogram frames of the object based audio program. A first container ofthe sequence of containers comprises a plurality of substream entitiesfor a plurality of substreams of the object based audio program.Furthermore, the first container comprises a presentation section. Themethod comprises determining a set of object channels indicative ofaudio content of at least some of a set of audio signals, wherein theset of object channels comprises a sequence of sets of object channelframes. The method also comprises providing or determining a set ofobject related metadata for the set of object channels, wherein the setof object related metadata comprises a sequence of sets of objectrelated metadata frames. A first audio program frame of the object basedaudio program comprises a first set of object channel frames of the setof object channel frames and a corresponding first set of object relatedmetadata frames. Furthermore, the method comprises inserting the firstset of object channel frames and the first set of object relatedmetadata frames into a respective set of object channel substreamentities of the plurality of substream entities of the first container.In addition, the method comprises inserting presentation data into thepresentation section, wherein the presentation data is indicative of atleast one presentation. The presentation comprises a set of substreamentities from the plurality of substream entities which are to bepresented simultaneously.

According to another aspect, a bitstream indicative of an object basedaudio program is described. The bitstream comprises a sequence ofcontainers for a corresponding sequence of audio program frames of theobject based audio program. A first container of the sequence ofcontainers comprises a first audio program frame of the object basedaudio program. The first audio program frame comprises a first set ofobject channel frames of a set of object channel frames and acorresponding first set of object related metadata frames. The set ofobject channel frames is indicative of audio content of at least some ofa set of audio signals. The first container comprises a plurality ofsubstream entities for a plurality of substreams of the object basedaudio program. The plurality of substream entities comprises a set ofobject channel substream entities for the first set of object channelframes, respectively. The first container further comprises apresentation section with presentation data, wherein the presentationdata is indicative of at least one presentation of the object basedaudio program. The presentation comprises a set of substream entitiesfrom the plurality of substream entities which are to be presentedsimultaneously.

According to another aspect, a method for generating a personalizedaudio program from a bitstream as outlined in the present document isdescribed. The method comprises extracting presentation data from thepresentation section, wherein the presentation data is indicative of apresentation for the personalized audio program, and wherein thepresentation comprises a set of substream entities from the plurality ofsubstream entities which are to be presented simultaneously.Furthermore, the method comprises extracting, based on the presentationdata, one or more object channel frames and corresponding one or moreobject related metadata frames from the set of object channel substreamentities of the first container.

According to a further aspect, a system (e.g. an encoder) for generatinga bitstream indicative of an object based audio program is described.The bitstream comprises a sequence of containers for a correspondingsequence of audio program frames of the object based audio program. Afirst container of the sequence of containers comprises a plurality ofsubstream entities for a plurality of substreams of the object basedaudio program. The first container further comprises a presentationsection. The system is configured to determine a set of object channelsindicative of audio content of at least some of a set of audio signals,wherein the set of object channels comprises a sequence of sets ofobject channel frames. Furthermore, the system is configured todetermine a set of object related metadata for the set of objectchannels, wherein the set of object related metadata comprises asequence of sets of object related metadata frames. A first audioprogram frame of the object based audio program comprises a first set ofobject channel frames of the set of object channel frames and acorresponding first set of object related metadata frames. In addition,the system is configured to insert the first set of object channelframes and the first set of object related metadata frames into arespective set of object channel substream entities of the plurality ofsubstream entities of the first container. Furthermore, the system isconfigured to insert presentation data into the presentation section,wherein the presentation data is indicative of at least onepresentation, and wherein the at least one presentation comprises a setof substream entities from the plurality of substream entities which areto be presented simultaneously.

According to another aspect, a system for generating a personalizedaudio program from a bitstream comprising an object based audio programis described. The bitstream is as described in the present document. Thesystem is configured to extract presentation data from the presentationsection, wherein the presentation data is indicative of a presentationfor the personalized audio program, and wherein the presentationcomprises a set of substream entities from the plurality of substreamentities which are to be presented simultaneously. Furthermore, thesystem is configured to extract, based on the presentation data, one ormore object channel frames and corresponding one or more object relatedmetadata frames from the set of object channel substream entities of thefirst container.

According to a further aspect, a software program is described. Thesoftware program may be adapted for execution on a processor and forperforming the method steps outlined in the present document whencarried out on the processor.

According to another aspect, a storage medium is described. The storagemedium may comprise a software program adapted for execution on aprocessor and for performing the method steps outlined in the presentdocument when carried out on the processor.

According to a further aspect, a computer program product is described.The computer program may comprise executable instructions for performingthe method steps outlined in the present document when executed on acomputer.

It should be noted that the methods and systems including its preferredembodiments as outlined in the present patent application may be usedstand-alone or in combination with the other methods and systemsdisclosed in this document. Furthermore, all aspects of the methods andsystems outlined in the present patent application may be arbitrarilycombined. In particular, the features of the claims may be combined withone another in an arbitrary manner.

SHORT DESCRIPTION OF THE FIGURES

The invention is explained below in an exemplary manner with referenceto the accompanying drawings, wherein

FIG. 1 shows a block diagram of an example audio processing chain;

FIG. 2 shows a block diagram of an example audio encoder;

FIG. 3 shows a block diagram of an example audio decoder;

FIG. 4 shows example presentation data and example substreams of anaudio program;

FIG. 5 shows an example structure of a bitstream comprising presentationdata; and

FIG. 6 shows a flow chart of an example method for generating abitstream comprising presentation data.

DETAILED DESCRIPTION

As indicated above, the present document is directed at the technicalproblem of providing a bitstream for a generic audio program whichallows a decoder of the bitstream to generate a personalized audioprogram from the bitstream in a resource efficient manner. Inparticular, the generation of the personalized audio program should beperformed with relatively low computational complexity. Furthermore, thebitstream which comprises the generic audio program should exhibit arelatively low bitrate.

FIG. 1 shows a block diagram of an example audio processing chain (alsoreferred to as an audio data processing system). The system includes thefollowings elements, coupled together as shown: a capture unit 1, aproduction unit 3 (which includes an encoding subsystem), a deliverysubsystem 5, a decoder 7, an object processing subsystem 9, a controller10, and a rendering subsystem 11. In variations on the system shown, oneor more of the elements are omitted, or additional audio data processingunits are included. Typically, elements 7, 9, 10, and 11 are included ina playback and/or decoding system (e.g., an end user's home theatersystem).

Capture unit 1 is typically configured to generate PCM (time-domain)samples comprising audio content, and to output the PCM samples. Thesamples may be indicative of multiple streams of audio captured bymicrophones (e.g., at a sporting event or other spectator event).Production unit 3, typically operated by a broadcaster, is configured toaccept the PCM samples as input and to output an object based audioprogram indicative of the audio content. The program typically is orincludes an encoded (e.g., compressed) audio bitstream indicative of theaudio content and presentation data which allows different personalizedaudio programs to be derived from the bitstream. The data of the encodedbitstream that are indicative of the audio content are sometimesreferred to herein as “audio data”. The object based audio programoutput from unit 3 may be indicative of (i.e., may include) multiplespeaker channels (a “bed” of speaker channels) of audio data, multipleobject channels of audio data, and object related metadata. The audioprogram may comprise presentation data which may be used to selectdifferent combinations of speaker channels and/or object channels inorder to generate different personalized audio programs (which may alsobe referred to as different experiences). By way of example, the objectbased audio program may include a main mix which in turn includes audiocontent indicative of a bed of speaker channels, audio contentindicative of at least one user-selectable object channel (and optionalat least one other object channel), and object related metadataassociated with each object channel. The program may also include atleast one side mix which includes audio content indicative of at leastone other object channel (e.g., at least one user-selectable objectchannel) and/or object related metadata. The audio program may beindicative of one or more beds, or no bed, of speaker channels. Forexample, the audio program (or a particular mix/presentation) may beindicative of two or more beds of speaker channels (e.g., a 5.1 channelneutral crowd noise bed, a 2.0 channel home team crowd noise bed, and a2.0 away team crowd noise bed), including at least one user-selectablebed (which can be selected using a user interface employed for userselection of object channel content or configuration) and a default bed(which will be rendered in the absence of user selection of anotherbed). The default bed may be determined by data indicative of aconfiguration (e.g., the initial configuration) of the speaker set ofthe playback system, and optionally the user may select another bed tobe rendered in place of the default bed.

Delivery subsystem 5 of FIG. 1 is configured to store and/or transmit(e.g., broadcast) the audio program generated by unit 3. Decoder 7accepts (receives or reads) the audio program delivered by deliverysubsystem 5, and decodes the program (or one or more accepted elementsthereof). Object processing subsystem 9 is coupled to receive (fromdecoder 7) decoded speaker channels, object channels, and object relatedmetadata of the delivered audio program. Subsystem 9 is coupled andconfigured to output to rendering subsystem 11 a selected subset of thefull set of object channels indicated by the audio program, andcorresponding object related metadata. Subsystem 9 is typically alsoconfigured to pass through unchanged (to subsystem 11) the decodedspeaker channels from decoder 7.

The object channel selection performed by subsystem 9 may be determinedby user selection(s) (as indicated by control data asserted to subsystem9 from controller 10) and/or rules (e.g., indicative of conditionsand/or constraints) which subsystem 9 has been programmed or otherwiseconfigured to implement. Such rules may be determined by object relatedmetadata of the audio program and/or by other data (e.g., dataindicative of the capabilities and organization of the playback system'sspeaker array) asserted to subsystem 9 (e.g., from controller 10 oranother external source) and/or by preconfiguring (e.g., programming)subsystem 9. Controller 10 (via a user interface implemented bycontroller 10) may provide (e.g., display on a touch screen) to the usera menu or palette of selectable “preset” mixes or presentation ofobjects and “bed” speaker channel content. The selectable preset mixesor presentations may be determined by presentation data comprised withinthe audio program and possibly also by rules implemented by subsystem 9(e.g., rules which subsystem 9 has been preconfigured to implement). Theuser selects from among the selectable mixes/presentations by enteringcommands to controller 10 (e.g., by actuating a touch screen thereof),and in response, controller 10 asserts corresponding control data tosubsystem 9.

Rendering subsystem 11 of FIG. 1 is configured to render the audiocontent determined by the output of subsystem 9, for playback by thespeakers (not shown) of the playback system. Subsystem 11 is configuredto map, to the available speaker channels, the audio objects determinedby the object channels selected by object processing subsystem 9 (e.g.,default objects, and/or user-selected objects which have been selectedas a result of user interaction using controller 10), using renderingparameters output from subsystem 9 (e.g., user-selected and/or defaultvalues of spatial position and level) which are associated with eachselected object. At least some of the rendering parameters may bedetermined by the object related metadata output from subsystem 9.Rendering system 11 also receives the bed of speaker channels passedthrough by subsystem 9. Typically, subsystem 11 is an intelligent mixer,and is configured to determine speaker feeds for the available speakersincluding by mapping one or more selected (e.g., default-selected)objects to each of a number of individual speaker channels, and mixingthe objects with “bed” audio content indicated by each correspondingspeaker channel of the program's speaker channel bed.

FIG. 2 is a block diagram of a broadcast system configured to generatean object based audio program (and a corresponding video program) forbroadcast. A set of X microphones (where X is an integer greater than 0,1 or 2), including microphones 100, 101, 102, and 103, of the FIG. 2system are positioned to capture audio content to be included in theaudio program, and their outputs are coupled to inputs of audio console104. The audio program may include interactive audio content which isindicative of the atmosphere in or at, and/or commentary on a spectatorevent (e.g., a soccer or rugby game, a car or motorcycle race, oranother sporting event). The audio program may comprise multiple audioobjects (including user-selectable objects or object sets, and typicallyalso a default set of objects to be rendered in the absence of objectselection by the user) and a mix (or “bed”) of speaker channels of theaudio program. The bed of speaker channels may be a conventional mix(e.g., a 5.1 channel mix) of speaker channels of a type that might beincluded in a conventional broadcast program which does not include anobject channel.

A subset of the microphones (e.g., microphones 100 and 101 andoptionally also other microphones whose outputs are coupled to audioconsole 104) may be a conventional array of microphones which, inoperation, captures audio (to be encoded and delivered as a bed ofspeaker channels). In operation, another subset of the microphones(e.g., microphones 102 and 103 and optionally also other microphoneswhose outputs are coupled to audio console 104) captures audio (e.g.,crowd noise and/or other “objects”) to be encoded and delivered asobject channels of the program. For example, the microphone array of theFIG. 2 system may include: at least one microphone (e.g., microphone100) implemented as a soundfield microphone and permanently installed ina stadium; at least one stereo microphone (e.g., microphone 102,implemented as a Sennheiser MKH416 microphone or another stereomicrophone) pointed at the location of spectators who support one team(e.g., the home team), and at least one other stereo microphone (e.g.,microphone 103, implemented as a Sennheiser MKH416 microphone or anotherstereo microphone) pointed at the location of spectators who support theother team (e.g., the visiting team).

The broadcasting system of FIG. 2 may include a mobile unit (which maybe a truck, and is sometimes referred to as a “match truck”) locatedoutside of a stadium (or other event location), which is the firstrecipient of audio feeds from microphones in the stadium (or other eventlocation). The match truck generates the object based audio program (tobe broadcast) including by encoding audio content from microphones fordelivery as object channels of the audio program, generatingcorresponding object related metadata (e.g., metadata indicative ofspatial location at which each object should be rendered) and includingsuch metadata in the audio program, and/or encoding audio content fromsome microphones for delivery as a bed of speaker channels of the audioprogram.

For example, in the FIG. 2 system, console 104, object processingsubsystem 106 (coupled to the outputs of console 104), embeddingsubsystem 108, and contribution encoder 110 may be installed in a matchtruck. The object based audio program generated in subsystem 106 may becombined (e.g., in subsystem 108) with video content (e.g., from cameraspositioned in the stadium) to generate a combined audio and video signalwhich is then encoded (e.g., by encoder 110), thereby generating anencoded audio/video signal for broadcast (e.g., by delivery subsystem 5of FIG. 1). It should be understood that a playback system which decodesand renders such an encoded audio/video signal would include a subsystem(not specifically shown in the drawings) for parsing the audio contentand the video content of the delivered audio/video signal, and asubsystem for decoding and rendering the audio content, and anothersubsystem (not specifically shown in the drawings) for decoding andrendering the video content.

The audio output of console 104 may include a 5.1 speaker channel bed(labeled “5.1 neutral” in FIG. 2) e.g. indicative of sound captured at asporting event, audio content of a stereo object channel (labeled “2.0home”) e.g. indicative of crowd noise from the home team's fans who arepresent at the event, audio content of a stereo object channel (labeled“2.0 away”) e.g. indicative of crowd noise from the visiting team's fanswho are present at the event, object channel audio content (labeled “1.0comm1”) e.g. indicative of commentary by an announcer from the hometeam's city, object channel audio content (labeled “1.0 comm2”) e.g.indicative of commentary by an announcer from the visiting team's city,and object channel audio content (labeled “1.0 ball kick”) e.g.indicative of sound produced by a game ball as it is struck by sportingevent participants.

Object processing subsystem 106 is configured to organize (e.g., group)audio streams from console 104 into object channels (e.g., to group theleft and right audio streams labeled “2.0 away” into a visiting crowdnoise object channel) and/or sets of object channels, to generate objectrelated metadata indicative of the object channels (and/or objectchannel sets), and to encode the object channels (and/or object channelsets), object related metadata, and the speaker channel bed (determinedfrom audio streams from console 104) as an object based audio program(e.g., an object based audio program encoded as an AC-4 bitstream).

Alternatively, the encoder 110 may be configured to generate the objectbased audio program, which may be encoded e.g. as an AC-4 bitstream. Insuch cases, the object processing subsystem 106 may be focused onproducing audio content (e.g. using a Dolby E+format), whereas theencoder 110 may be focused on generating a bitstream for emission ordistribution.

Subsystem 106 may further be configured to render (and play on a set ofstudio monitor speakers) at least a selected subset of the objectchannels (and/or object channel sets) and the speaker channel bed(including by using the object related metadata to generate amix/presentation indicative of the selected object channel(s) andspeaker channels) so that the played back sound can be monitored by theoperator(s) of console 104 and subsystem 106 (as indicated by the“monitor path” of FIG. 2).

The interface between subsystem 104's outputs and subsystem 106's inputsmay be a multichannel audio digital interface (“MADI”).

In operation, subsystem 108 of the FIG. 2 system may combine the objectbased audio program generated in subsystem 106 with video content (e.g.,from cameras positioned in a stadium) to generate a combined audio andvideo signal which is asserted to encoder 110. The interface betweensubsystem 108's output and subsystem 110's input may be a highdefinition serial digital interface (“HD-SDI”). In operation, encoder110 encodes the output of subsystem 108, thereby generating an encodedaudio/video signal for broadcast (e.g., by delivery subsystem 5 of FIG.1).

A broadcast facility (e.g., subsystems 106, 108, and 110 of the FIG. 2system) may be configured to generate different presentations ofelements of an object based audio program. Examples of suchpresentations comprise a 5.1 flattened mix, an international mix, and adomestic mix. For example, all the presentations may include a commonbed of speaker channels, but the object channels of the presentations(and/or the menu of selectable object channels determined by thepresentations, and/or selectable or non-selectable rendering parametersfor rendering and mixing the object channels) may differ frompresentation to presentation.

Object related metadata of an audio program (or a preconfiguration ofthe playback or rendering system, not indicated by metadata deliveredwith the audio program) may provide constraints or conditions onselectable mixes/presentations of objects and bed (speaker channel)content. For example, a DRM hierarchy may be implemented to allow a userto have tiered access to a set of object channels included in an objectbased audio program. If the user pays more money (e.g., to thebroadcaster), the user may be authorized to decode, select, and rendermore object channels of the audio program.

FIG. 3 is a block diagram of an example playback system which includesdecoder 20, object processing subsystem 22, spatial rendering subsystem24, controller 23 (which implements a user interface), and optionallyalso digital audio processing subsystems 25, 26, and 27, coupled asshown. In some implementations, elements 20, 22, 24, 25, 26, 27, 29, 31,and 33 of the FIG. 3 system are implemented as a set top device.

In the system of FIG. 3, decoder 20 is configured to receive and decodean encoded signal indicative of object based audio program. The audioprogram is indicative of audio content including e.g. two speakerchannels (i.e., a “bed” of at least two speaker channels). The audioprogram is also indicative of at least one user-selectable objectchannel (and optionally at least one other object channel) and objectrelated metadata corresponding to each object channel. Each objectchannel is indicative of an audio object, and thus object channels aresometimes referred to herein as “objects” for convenience. The audioprogram may be comprised within an AC-4 bitstream, indicative of audioobjects, object-related metadata, and/or a bed of speaker channels.Typically, the individual audio objects are either mono or stereo coded(i.e., each object channel is indicative of a left or right channel ofan object, or is a monophonic channel indicative of an object), the bedmay be a traditional 5.1 mix, and decoder 20 may be configured to decodea pre-determined number (e.g. 16 or more) of channels of audio content(including e.g. six speaker channels of the bed, and e.g. ten or moreobject channels) simultaneously. The incoming bitstream may beindicative of a certain number of (e.g. more than ten) audio objects,and not all of them may need to be decoded to achieve a specificmix/presentation.

As indicated above, the audio program may comprise zero, one or morebeds of speaker channels as well as one or more object channels. A bedof speaker channels and/or an object channel may form a substream of thebitstream which comprises the audio program. Hence, the bitstream maycomprise a plurality of substreams, wherein a substream is indicative ofa bed of speaker channels or of one or more object channels.Furthermore, the bitstream may comprise presentation data (e.g.comprised within a presentation section of the bitstream), wherein thepresentation data may be indicative of one or more differentpresentations. A presentation may define a particular mix of substreams.In other words, a presentation may define a bed of speaker channelsand/or one or more object channels which are to be mixed together inorder to provide a personalized audio program.

FIG. 4 illustrates a plurality of substreams 411, 412, 413, 414. Eachsubstream 411, 412, 413, 414 comprises audio data 421, 424, wherein theaudio data 421, 424 may correspond to a bed of speaker channels or tothe audio data of an audio object (i.e. to an audio channel). By way ofexample, the substream 411 may comprise a bed of speaker channels 421and the substream 414 may comprise an object channel 424. Furthermore,each substream 411, 412, 413, 414 may comprise metadata 431, 434 (e.g.default metadata) which is associated with the audio data 421, 424 andwhich may be used for rendering the associated audio data 421, 424. Byway of example, the substream 411 may comprise speaker related metadata(for a bed of speaker channels 421) and the substream 414 may compriseobject related metadata (for an object channel 424). In addition, asubstream 411, 412, 413, 414 may comprise alternative metadata 441, 444,in order to provide one or more alternative ways for rendering theassociated audio data 421, 424.

Furthermore, FIG. 4 shows different presentations 401, 402, 403. Apresentation 401 is indicative of a selection of substreams 411, 412,413, 414 which are to be used for the presentation 401, thereby defininga personalized audio program. Furthermore, a presentation 401 may beindicative of the metadata 431, 441 (e.g. the default metadata 431 orone of the alternative metadata 441) which is to be used for a selectedsubstream 411 for the presentation 401. In the illustrated example, thepresentation 401 describes a personalized audio program which comprisesthe substreams 411, 412, 414.

As such, the use of presentations 401, 402, 403 provides an efficientmeans for signaling different personalized audio programs within ageneric object based audio program. In particular, the presentations401, 402, 403 may be such that a decoder 7, 20 can easily select the oneor more substreams 411, 412, 413, 414 which are required for aparticular presentation 401, without the need for decoding the completebitstream of the generic object based audio program. Furthermore, are-multiplexer (not shown in FIG. 3) may be configured to easily extractthe one or more substreams 411, 412, 413, 414 from the completebitstream, in order to generate a new bitstream for the personalizedaudio program of the particular presentation 401. In other words, from abitstream with a relatively large number of presentations 401, 402, 403,a new bitstream may be generated efficiently that carries a reducednumber of presentations. A possible scenario is a relatively largebitstream with a relatively high number of presentations arriving at anSTB. The STB may be focused on personalization (i.e. on selecting apresentation) and may be configured to repackage a single-presentationbitstream (without decoding the audio data). The single-presentationbitstream (and the audio data) may then be decoded at an appropriate,remote decoder, e.g. within an AVR (Audio/Video receiver) or within amobile home device such as a tablet PC.

The decoder (e.g., decoder 20 of FIG. 3) may parse the presentation datain order to identify a presentation 401 for rendering. Furthermore, thedecoder 200 may extract the substreams 411, 412, 414 which are requiredfor the presentation 401 from the locations indicated by thepresentation data. After extracting the substreams 411, 412, 414(speaker channels, object channels and associated metadata), the decodermay perform any necessary decoding (e.g. only) on the extractedsubstreams 411, 412, 414.

The bitstream may be an AC-4 bitstream and the presentations 401, 402,403 may be AC-4 presentations. These presentations enable an easy accessto the parts of a bitstream (audio data 421 and metadata 431) that areneeded for a particular presentation. In that way, a decoder or receiversystem 20 can easily access the needed parts of a bitstream without theneed to parse deeply into other parts of a bitstream. This for instancealso enables the possibility to only forward the required parts of abitstream to another device without the need to re-build the wholestructure or even to decode and encode substreams 411, 412, 413, 414 ofthe bitstream. In particular, a reduced structure which is derived fromthe bitstream may be extracted.

With reference again to FIG. 3, a user may employ controller 23 toselect objects (indicated by the object based audio program) to berendered. By way of example, the user may select a particularpresentation 401. Controller 23 may be a handheld processing device(e.g., an iPad) which is programmed to implement a user interface (e.g.,an iPad App) compatible with the other elements of the FIG. 3 system.The user interface may provide (e.g., display on a touch screen) to theuser a menu or palette of selectable presentations 401, 402, 403 (e.g.“preset” mixes) of objects and/or “bed” speaker channel content. Thepresentations 401, 402, 403 may be provided along with a name tag withinthe menu or palette. The selectable presentations 401, 402, 403 may bedetermined by presentation data of the bitstream and possibly also byrules implemented by subsystem 22 (e.g., rules which subsystem 22 hasbeen preconfigured to implement). The user may select from among theselectable presentations by entering commands to controller 23 (e.g., byactuating a touch screen thereof), and in response, controller 23 mayassert corresponding control data to subsystem 22.

In response to the object based audio program, and in response tocontrol data from controller 23 indicative of the selected presentation401, decoder 20 decodes (if necessary) the speaker channels of the bedof speaker channels of a selected presentation 401, and outputs tosubsystem 22 decoded speaker channels. In response to the object basedaudio program, and in response to control data from controller 23indicative of the selected presentation 401, decoder 20 decodes (ifnecessary) the selected object channels, and outputs to subsystem 22 theselected (e.g., decoded) object channels (each of which may be a pulsecode modulated or “PCM” bitstream), and object related metadatacorresponding to the selected object channels.

The objects indicated by the decoded object channels typically are orinclude user-selectable audio objects. For example, as indicated in FIG.3, the decoder 20 may extract a 5.1 speaker channel bed, an objectchannel (“Comment-1 mono”) indicative of commentary by an announcer fromthe home team's city, an object channel (“Comment-2 mono”) indicative ofcommentary by an announcer from the visiting team's city, an objectchannel (“Fans (home)”) indicative of crowd noise from the home team'sfans who are present at a sporting event, left and right object channels(“Ball Sound stereo”) indicative of sound produced by a game ball as itis struck by sporting event participants, and four object channels(“Effects 4x mono”) indicative of special effects. Any of the “Comment-1mono,” “Comment-2 mono,” “Fans (home),” “Ball Sound stereo,” and“Effects 4x mono” object channels may be selected as part of apresentation 401, and each selected one of them would be passed fromsubsystem 22 to rendering subsystem 24 (after undergoing any necessarydecoding in decoder 20).

Subsystem 22 is configured to output a selected subset of the full setof object channels indicated by the audio program, and correspondingobject related metadata of the audio program. The object selection maybe determined by user selections (as indicated by control data assertedto subsystem 22 from controller 23) and/or rules (e.g., indicative ofconditions and/or constraints) which subsystem 22 has been programmed orotherwise configured to implement. Such rules may be determined byobject related metadata of the program and/or by other data (e.g., dataindicative of the capabilities and organization of the playback system'sspeaker array) asserted to subsystem 22 (e.g., from controller 23 oranother external source) and/or by preconfiguring (e.g., programming)subsystem 22. As indicated above, the bitstream may comprisepresentation data providing a set of selectable “preset” mixes (i.e.presentations 401, 402, 403) of objects and “bed” speaker channelcontent. Subsystem 22 typically passes through unchanged (to subsystem24) the decoded speaker channels from decoder 20, and processes selectedones of the object channels asserted thereto.

Spatial rendering subsystem 24 of FIG. 3 (or subsystem 24 with at leastone downstream device or system) is configured to render the audiocontent output from subsystem 22 for playback by speakers of the user'splayback system. One or more of optionally included digital audioprocessing subsystems 25, 26, and 27 may implement post-processing onthe output of subsystem 24.

Spatial rendering subsystem 24 is configured to map, to the availablespeaker channels, the audio object channels selected by objectprocessing subsystem 22, using rendering parameters output fromsubsystem 22 (e.g., user-selected and/or default values of spatialposition and level) which are associated with each selected object.Spatial rendering system 24 also receives a decoded bed of speakerchannels passed through by subsystem 22. Typically, subsystem 24 is anintelligent mixer, and is configured to determine speaker feeds for theavailable speakers including by mapping one, two, or more than twoselected object channels to each of a number of individual speakerchannels, and mixing the selected object channel(s) with “bed” audiocontent indicated by each corresponding speaker channel of the program'sspeaker channel bed.

The speakers to be driven to render the audio may be located inarbitrary locations in the playback environment; not merely in a(nominally) horizontal plane. In some such cases, metadata included inthe program indicates rendering parameters for rendering at least oneobject of the program at any apparent spatial location (in a threedimensional volume) using a three-dimensional array of speakers. Forexample, an object channel may have corresponding metadata indicating athree-dimensional trajectory of apparent spatial positions at which theobject (indicated by the object channel) is to be rendered. Thetrajectory may include a sequence of “floor” locations (in the plane ofa subset of speakers which are assumed to be located on the floor, or inanother horizontal plane, of the playback environment), and a sequenceof “above-floor” locations (each determined by driving a subset of thespeakers which are assumed to be located in at least one otherhorizontal plane of the playback environment). In such cases, therendering can be performed in accordance with the present invention sothat the speakers can be driven to emit sound (determined by therelevant object channel) that will be perceived as emitting from asequence of object locations in the three-dimensional space whichincludes the trajectory, mixed with sound determined by the “bed” audiocontent. Subsystem 24 may be configured to implement such rendering, orsteps thereof, with remaining steps of the rendering being performed bya downstream system or device (e.g., rendering subsystem 35 of FIG. 3).

Optionally, a digital audio processing (DAP) stage (e.g., one for eachof a number of predetermined output speaker channel configurations) iscoupled to the output of spatial rendering subsystem 24 to performpost-processing on the output of the spatial rendering subsystem.Examples of such processing include intelligent equalization or (in caseof a stereo output) speaker virtualization processing.

The output of the FIG. 3 system (e.g., the output of the spatialrendering subsystem, or a DAP stage following the spatial renderingstage) may be PCM bitstreams (which determine speaker feeds for theavailable speakers). For example, in the case that the user's playbacksystem includes a 7.1 array of speakers, the system may output PCMbitstreams (generated in subsystem 24) which determine speaker feeds forthe speakers of such array, or a post-processed version (generated inDAP 25) of such bitstreams. For another example, in the case that theuser's playback system includes a 5.1 array of speakers, the system mayoutput PCM bitstreams (generated in subsystem 24) which determinespeaker feeds for the speakers of such array, or a post-processedversion (generated in DAP 26) of such bitstreams. For another example,in the case that the user's playback system includes only left and rightspeakers, the system may output PCM bitstreams (generated in subsystem24) which determine speaker feeds for the left and right speakers, or apost-processed version (generated in DAP 27) of such bitstreams.

The FIG. 3 system optionally also includes one or both of re-encodingsubsystems 31 and 33. Re-encoding subsystem 31 is configured tore-encode the PCM bitstream (indicative of feeds for a 7.1 speakerarray) output from DAP 25 as an encoded bitstream (e.g. an AC-4 or AC-3bitstream), and the resulting encoded (compressed) AC-3 bitstream may beoutput from the system. Re-encoding subsystem 33 is configured tore-encode the PCM bitstream (indicative of feeds for a 5.1 speakerarray) output from DAP 27 as an encoded bitstream (e.g. an AC-4 or AC-3bitstream), and the resulting encoded (compressed) bitstream may beoutput from the system.

The FIG. 3 system optionally also includes re-encoding (or formatting)subsystem 29 and downstream rendering subsystem 35 coupled to receivethe output of subsystem 29. Subsystem 29 is coupled to receive data(output from subsystem 22) indicative of the selected audio objects (ordefault mix of audio objects), corresponding object related metadata,and the bed of speaker channels, and is configured to re-encode (and/orformat) such data for rendering by subsystem 35. Subsystem 35, which maybe implemented in an AVR or soundbar (or other system or devicedownstream from subsystem 29), is configured to generate speaker feeds(or bitstreams which determine speaker feeds) for the available playbackspeakers (speaker array 36), in response to the output of subsystem 29.For example, subsystem 29 may be configured to generate encoded audio,by re-encoding the data indicative of the selected (or default) audioobjects, corresponding metadata, and bed of speaker channels, into asuitable format for rendering in subsystem 35, and to transmit theencoded audio (e.g., via an HDMI link) to subsystem 35. In response tospeaker feeds generated by (or determined by the output of) subsystem35, the available speakers 36 would emit sound indicative of a mix ofthe speaker channel bed and the selected (or default) object(s), withthe object(s) having apparent source location(s) determined by objectrelated metadata of subsystem 29's output. When subsystems 29 and 35 areincluded, rendering subsystem 24 is optionally omitted from the system.

As indicated above, the use of presentation data is beneficial as itenables the decoder 20 to efficiently select the one or more substreams411, 412, 413, 414 which are required for a particular presentation 401.In view of this, the decoder 20 may be configured to extract the one ormore substreams 411, 412, 413, 414 of a particular presentation 401, andto rebuild a new bitstream which (typically only) comprises the one ormore substreams 411, 412, 413, 414 of the particular presentation 401.This extraction and re-building of a new bitstream may be performedwithout the need for actually decoding and re-encoding the one or moresubstreams 411, 412, 413, 414. Hence, the generation of the newbitstream for the particular presentation 401 may be performed in aresource efficient manner.

The system of FIG. 3 may be a distributed system for rendering objectbased audio, in which a portion (i.e., at least one step) of therendering (e.g., selection of audio objects to be rendered and selectionof characteristics of the rendering of each selected object, asperformed by subsystem 22 and controller 23 of the FIG. 3 system) isimplemented in a first subsystem (e.g., elements 20, 22, and 23 of FIG.3, implemented in a set top device, or a set top device and a handheldcontroller) and another portion of the rendering (e.g., immersiverendering in which speaker feeds, or signals which determine speakerfeeds, are generated in response to the output of the first subsystem)is implemented in a second subsystem (e.g., subsystem 35, implemented inan AVR or soundbar). Latency management may be provided to account forthe different times at which and different subsystems in which portionsof the audio rendering (and any processing of video which corresponds tothe audio being rendered) are performed.

As illustrated in FIG. 5, the generic audio program may be transportedin a bitstream 500 which comprises a sequence of containers 501. Eachcontainer 501 may comprise data of the audio program for a particularframe of the audio program. A particular frame of the audio program maycorrespond to a particular temporal segment of the audio program (e.g.20 milliseconds of the audio program). Hence, each container 501 of thesequence of containers 501 may carry the data for a frame of a sequenceof frames of the generic audio program. The data for a frame may becomprised within a frame entity 502 of a container 501. The frame entitymay be identified using a syntax element of the bitstream 500.

As indicated above, the bitstream 500 may carry a plurality ofsubstreams 411, 412, 413, 414, wherein each substream 411 comprises abed of speaker channels 421 or an object channel 424. As such, a frameentity 502 may comprise a plurality of corresponding substream entities520. Furthermore, a frame entity 502 may comprise a presentation section510 (also referred to as a Table of Content, TOC, section). Thepresentation section 510 may comprise TOC data 511 which may indicatee.g. a number of presentations 401, 402, 403 comprised within thepresentation section 510. Furthermore, the presentation section 510 maycomprise one or more presentation entities 512 which carry data fordefining one or more presentations 401, 402, 403, respectively. Asubstream entity 520 may comprise a content sub-entity 521 for carryingthe audio data 421, 424 of a frame of a substream 411. Furthermore, asubstream entity 520 may comprise a metadata sub-entity 522 for carryingthe corresponding metadata 431, 441 of the frame of the substream 411.

FIG. 6 shows a flow chart of an example method 600 for generating abitstream 500 which is indicative of an object based audio program (i.e.of a generic audio program). The bitstream 500 exhibits a bitstreamformat, such that the bitstream 500 comprises a sequence of containers501 for a corresponding sequence of audio program frames of the objectbased audio program. In other words, each frame (i.e. each temporalsegment) of the object based audio program may be inserted into acontainer of the sequence of containers, which may be defined by thebitstream format. A container may be defined using a particularcontainer syntax element of the bitstream format. By way of example, thebitstream format may correspond to an AC-4 bitstream format. In otherwords, the to-be-generated bitstream 500 may be an AC-4 bitstream.

Furthermore, the bitstream format may be such that a first container 501of the sequence of containers 501 (i.e. at least one of the containers501 of the sequence of containers 501) comprises a plurality ofsubstream entities 520 for a plurality of substreams 411, 412, 413, 414of the object based audio program. As outlined above, an audio programmay comprise a plurality of substreams 411, 412, 413, 414, wherein eachsubstream 411, 412, 413, 414 may either comprise a bed of speakerchannels 421 or an object channel 424 or both. The bitstream format maybe such that each container 501 of the sequence of containers 501provides a dedicated substream entity 520 for a corresponding substream411, 412, 413, 414. In particular, each substream entity 520 maycomprise the data relating to the frame of a corresponding substream411, 412, 413, 414. The frame of a substream 411, 412, 413, 414 may bethe frame of a bed of speaker channels 421, which is referred to hereinas a speaker channel frame. Alternatively, the frame of a substream 411,412, 413, 414 may be the frame of an object channel, which is referredto herein as an object channel frame. A substream entity 520 may bedefined by a corresponding syntax element of the bitstream format.

Furthermore, the first container 501 may comprise a presentation section510. In other words, the bitstream format may allow for the definitionof a presentation section 510 (e.g. using an appropriate syntax element)for all of the containers 501 of a sequence of containers 501. Thepresentation section 510 may be used for defining differentpresentations 401, 402, 403 for different personalized audio programsthat can be generated from the (generic) object based audio program.

The method 600 comprises determining 601 a set of object channels 424which are indicative of audio content of at least some of a set of audiosignals. The set of audio signals may be indicative of captured audiocontent, e.g. audio content which has been captured using a systemdescribed in the context of FIG. 2. The set of object channels 424 maycomprise a plurality of object channels 424. Furthermore, the set ofobject channels 424 comprises a sequence of sets of object channelframes. In other words, each object channel comprises a sequence ofobject channel frames. By consequence, the set of object channelscomprises a sequence of sets of object channel frames, wherein a set ofobject channel frames at a particular time instant comprises the objectchannel frames of the set of object channels at the particular timeinstant.

Furthermore, the method 600 comprises providing or determining 602 a setof object related metadata 434, 444 for the set of object channels 424,wherein the set of object related metadata 434, 444 comprises a sequenceof sets of object related metadata frames. In other words, the objectrelated metadata of an object channel is segmented into a sequence ofobject related metadata frames. By consequence, the set of objectrelated metadata for the corresponding set of object channels comprisesa sequence of sets of object related metadata frames.

As such, an object related metadata frame may be provided for acorresponding object channel frame (e.g. using the object processor 106described in the context of FIG. 2). As indicated above, an objectchannel 424 may be provided with different variants of object relatedmetadata 434, 444. By way of example, a default variant of objectrelated metadata 434 and one or more alternative variants of objectrelated metadata 444 may be provided. By doing this, differentperspectives (e.g. different positions within a stadium) may besimulated. Alternatively or in addition, a bed of speaker channels 421may be provided with different variants of speaker related metadata 431,441. By way of example, a default variant of speaker related metadata431 and one or more alternative variants of speaker related metadata 441may be provided. By doing this, different rotations of the bed ofspeaker channels 421 may be defined. Similar to the object relatedmetadata also the speaker related metadata may be time-variant.

As such, an audio program may comprise a set of object channels. Byconsequence, a first audio program frame of the object based audioprogram may comprise a first set of object channel frames from thesequence of sets of object channel frames and a corresponding first setof object related metadata frames from the sequence of sets of objectrelated metadata frames.

The method 600 may further comprise inserting 603 the first set ofobject channel frames and the first set of object related metadataframes into a respective set of object channel substream entities 520 ofthe plurality of substream entities 520 of the first container 501. Assuch, a substream 411, 412, 413, 414 may be generated for each objectchannel 421 of the object based audio program. Each substream 411, 412,413, 414 may be identified within the bitstream 500 via the respectivesubstream entity 520 which carries the substream 411, 412, 413, 414. Asa result of this, different substreams 411, 412, 413, 414 may beidentified and possibility extracted by a decoder 7, 20 in a resourceefficient manner, without the need for decoding the complete bitstream500 and/or the substreams 411, 412, 413, 414.

Furthermore, the method 600 comprises inserting 604 presentation datainto the presentation section 510 of the bitstream 500. The presentationdata may be indicative of at least one presentation 401, wherein the atleast one presentation 401 may define a personalized audio program. Inparticular, the at least one presentation 401 may comprise or mayindicate a set of substream entities 520 from the plurality of substreamentities 520 which are to be presented simultaneously. As such, apresentation 401 may indicate which one or more of the substreams 411,412, 413, 414 of an object based audio program are to be selected forgenerating a personalized audio program. As outlined above, apresentation 401 may identify a subset of the complete set of substreams411, 412, 413, 414 (i.e. less than the total number of substreams 411,412, 413, 414).

The insertion of presentation data enables a corresponding decoder 7, 20to identify and to extract one or more substreams 411, 412, 413, 414from the bitstream 500 to generate a personalized audio program withoutthe need for decoding or parsing the complete bitstream 500.

The method 600 may comprise determining a bed of speaker channels 421which is indicative of audio content of one or more of the set of audiosignals. The bed of speaker channels 421 may comprise one or more of:2.0 channels, 5.1 channels, 5.1.2 channels, 7.1 channels and/or 7.1.4channels. A bed of speaker channels 421 may be used to provide a basisfor a personalized audio program. In addition, one or more objectchannels 424 may be used to provide personalized variations of thepersonalized audio program.

A bed of speaker channels 421 may comprise a sequence of speaker channelframes, and the first audio program frame of the object based audioprogram may comprise a first speaker channel frame of the sequence ofspeaker channel frames. The method 600 may further comprise insertingthe first speaker channel frame into a speaker channel substream entity520 of the plurality of substreams entities 520 of the first container501. A presentation 401 of the presentation section 510 may thencomprise or indicate the speaker channel substream entity 520.Alternatively or in addition, a presentation 401 may comprise or mayindicate one or more object channel substream entities 520 from the setof object channel substream entities.

The method 600 may further comprise providing speaker related metadata431, 441 for the bed of speaker channels 421. The speaker relatedmetadata 431, 441 may comprise a sequence of speaker related metadataframes. A first speaker related metadata frame from the sequence ofspeaker related metadata frames may be inserted into the speaker channelsubstream entity 520. It should be noted that a plurality of beds ofspeaker channels 421 may be inserted into a corresponding plurality ofspeaker channel substream entities 520.

As outlined in the context of FIG. 4, the presentation data may beindicative of a plurality of presentations 401, 402, 403 comprisingdifferent sets of substream entities 520 for different personalizedaudio programs. The different sets of substream entities 520 maycomprise different combinations of the one or more speaker channelsubstream entities 520, the one or more object channel substreamentities 520 and/or different combinations of variants of metadata 434,444 (e.g. default metadata 434 or alternative metadata 444).

The presentation data within the presentation section 510 may besegmented into different presentation data entities 512 for differentpresentations 401, 402, 403 (e.g. using an appropriate syntax element ofthe bitstream format). The method 600 may further comprise insertingtable of content (TOC) data into the presentation section 510. The TOCdata may be indicative of a position of the different presentation dataentities 512 within the presentation section 510 and/or an identifierfor the different presentations 401, 402, 403 comprised within thepresentation section 510. As such, the TOC data may be used by acorresponding decoder 7, 20 to identify and extract the differentpresentations 401, 402, 403 in an efficient manner Alternatively or inaddition, the presentation data entities 512 for the differentpresentations 401, 402, 403 may be comprised sequentially within thepresentation section 510. If the TOC data is not indicative of thepositions of the different presentation data entities 512, acorresponding decoder 7, 20 may identify and extract the differentpresentations 401, 402, 403 by parsing sequentially through thedifferent presentation data entities 512. This may be a bit-rateefficient method for signaling the different presentations 401, 402,403.

A substream entity 520 may comprise a content sub-entity 521 for audiocontent or audio data 424 and a metadata sub-entity 522 for relatedmetadata 434, 444. The sub-entities 521, 522 may be identified byappropriate syntax elements of the bitstream format. By doing this, acorresponding decoder 7, 20 may identify the audio data and thecorresponding metadata of an object channel or of a bed of speakerchannels in a resource efficient manner

As already indicated above, a metadata frame for a corresponding channelframe may comprise a plurality of different variants or groups 434, 444of metadata. A presentation 401 may be indicative of which variant orgroup 434 of metadata is to be used for rendering the correspondingchannel frame. By doing this, a degree of personalization of an audioprogram (e.g. a listening/viewing perspective) may be increased.

A bed of speaker channels 421 typically comprises one or more speakerchannels to be presented by one or more speakers 36, respectively, of apresentation environment. On the other hand, an object channel 424 istypically to be presented by a combination of speakers 36 of thepresentation environment. The object related metadata 434, 444 of anobject channel 424 may be indicative of a position within thepresentation environment from which the object channel 424 is to berendered. The position of the object channel 424 may be time-varying. Asa result of this, a combination of speakers 36 for rendering the objectchannel 424 may change along the sequence of object channel frames of anobject channel 424 and/or a panning of the speakers 36 of thecombination of speakers may change along the sequence of object channelframes of the object channel 424.

A presentation 401, 402, 403 may comprise target device configurationdata for a target device configuration. In other words, a presentation401, 402, 403 may be dependent on the target device configuration whichis used for rendering of the presentation 401, 402, 403. Target deviceconfigurations may differ with regards to the number of speakers, thepositions of the speakers and/or with regards to the number of audiochannels which may be processed and rendered. Example target deviceconfigurations are a 2.0 (stereo) target device configuration with aleft and a right loudspeaker, or a 5.1 target device configuration, etc.The target device configuration typically comprises a spatial renderingsubsystem 24, as described in the context of FIG. 3.

As such, a presentation 401, 402, 403 may indicate different audioresources to be used for different target device configurations. Thetarget device configuration data may be indicative of a set of substreamentities 520 from the plurality of substream entities 520 and/or of avariant 434 of metadata, which are to be used for rendering thepresentation 401 on a particular target device configuration. Inparticular, the target device configuration data may indicate suchinformation for a plurality of different target device configurations.By way of example, a presentation 401 may comprise different sectionswith target device configuration data for different target deviceconfigurations.

By doing this, a corresponding decoder or de-multiplexer may efficientlyidentify the audio resources (one or more substreams 411, 412, 413, 414,one or more variants 441 of metadata) which are to be used for aparticular target device configuration.

The bitstream format may allow for a further (intermediate) layer fordefining a personalized audio program. In particular, the bitstreamformat may allow for the definition of a substream group which comprisesone, two or more of the plurality of substreams 411, 412, 413, 414. Asubstream group may be used to group different audio content such asatmospheric content, dialogs and/or effects. A presentation 401 may beindicative of a substream group. In other words, a presentation 401 mayidentify one, two or more substreams, which are to be renderedsimultaneously, by referring to a substream group which comprises theone, two or more substreams. As such, substream groups provide anefficient means for identifying two or more substreams (which arepossibly associated with one another).

The presentation section 510 may comprise one or more substream groupentities (not shown in FIG. 5) for defining one or more correspondingsubstream groups. The substream group entities may be positionedsubsequent to or downstream of the presentation data entities 512. Asubstream group entity may be indicative of one or more substreams 411,412, 413, 414 which are comprised within the corresponding substreamgroup. A presentation 401 (defined within a corresponding presentationdata entity 512) may be indicative of the substream group entity, inorder to include the corresponding substream group into the presentation401. A decoder 7, 20 may parse through the presentation data entities512 for identifying a particular presentation 401. If the presentation401 makes reference to a substream group or to a substream group entity,the decoder 7, 20 may continue to parse through the presentation section510 to identify the definition of the substream group comprised within asubstream group entity of the presentation section 510. Hence, thedecoder 7, 20 may determine the substreams 411, 412, 413, 414 for aparticular presentation 401 by parsing through the presentation dataentities 512 and through the substream group entities of thepresentation section 510.

As such, the method 600 for generating a bitstream 500 may compriseinserting data for identifying the one, two or more of the plurality ofsubstreams into a substream group entity of the presentation section510. As a result of this, the substream group entity comprises data fordefining the substream group.

The definition of a substream group may be beneficial in view ofbit-rate reduction. In particular, a plurality of substreams 411, 412,413, 414 which are used jointly within a plurality of presentations 401,402, 403 may be grouped within a substream group. As a result of this,the plurality of substreams 411, 412, 413, 414 may be identifiedefficiently within the presentations 401, 402, 403 by referring to thesubstream group. Furthermore, the definition of a substream group mayprovide an efficient means for a content designer to master acombination of substreams 411, 412, 413, 414 and to define a substreamgroup for the mastered combination of substreams 411, 412, 413, 414.

Hence, a bitstream 500 is described, which is indicative of an objectbased audio program and which allows for a resource efficientpersonalization. The bitstream 500 comprises a sequence of containers501 for a corresponding sequence of audio program frames of the objectbased audio program, wherein a first container 501 of the sequence ofcontainers 501 comprises a first audio program frame of the object basedaudio program. The first audio program frame comprises a first set ofobject channel frames of a set of object channels and a correspondingfirst set of object related metadata frames. The set of object channelsmay be indicative of audio content of at least some of a set of audiosignals. Furthermore, the first container 501 comprises a plurality ofsubstream entities 520 for a plurality of substreams 411, 412, 413, 414of the object based audio program, wherein the plurality of substreamentities 520 comprises a set of object channel substream entities 520for the first set of object channel frames, respectively. The firstcontainer 501 further comprises a presentation section 510 withpresentation data. The presentation data may be indicative of at leastone presentation 401 of the object based audio program, wherein the atleast one presentation 401 comprises a set of substream entities 520from the plurality of substream entities 520 which are to be presentedsimultaneously.

The first audio program frame may further comprise a first speakerchannel frame of a bed of speaker channels 421, wherein the bed ofspeaker channels 421 is indicative of audio content of one or more ofthe set of audio signals. The plurality of substream entities 520 of thebitstream 500 may then comprise a speaker channel substream entity 520for the first speaker channel frame.

The bitstream 500 may be received by a decoder 7, 20. The decoder 7, 20may be configured to execute a method for generating a personalizedaudio program from the bitstream 500. The method may comprise extractingpresentation data from the presentation section 510. As indicated above,the presentation data may be indicative of a presentation 401 for thepersonalized audio program. Furthermore, the method may compriseextracting, based on the presentation data, one or more object channelframes and corresponding one or more object related metadata frames fromthe set of object channel substream entities 520 of the first container501, in order to generate and/or render the personalized audio program.Depending on the content of the bitstream, the method may furthercomprise extracting, based on the presentation data, the first speakerchannel frame from the speaker channel substream entity 520 of the firstcontainer 501.

The methods and bitstreams described in the present document arebeneficial in view of the creation of personalized audio programs for ageneric object based audio program. In particular, the described methodsand bitstreams allow parts of a bitstream to be stripped off orextracted in a resource efficient manner. By way of example, if onlyparts of a bitstream need to be forwarded, this may be done withoutforwarding/processing the full set of metadata and/or the full set ofaudio data. Only the required parts of the bitstream need to beprocessed and forwarded. A decoder may only be required to parse thepresentation section (e.g. the TOC data) of a bitstream in order toidentify the content comprised within the bitstream. Furthermore, abitstream may provide a “default” presentation (e.g. a “standard mix”)which can be used by a decoder to start rendering of a program withoutfurther parsing. In addition, a decoder only needs to decode the partsof a bitstream which are required for rendering a particularpersonalized audio program. This is achieved by an appropriateclustering of audio data into substreams and substream entities. Theaudio program may comprise a possibly unlimited number of substreams andsubstream entities, thereby providing a bitstream format with a highdegree of flexibility.

The methods and systems described in the present document may beimplemented as software, firmware and/or hardware. Certain componentsmay e.g. be implemented as software running on a digital signalprocessor or microprocessor. Other components may e.g. be implemented ashardware and or as application specific integrated circuits. The signalsencountered in the described methods and systems may be stored on mediasuch as random access memory or optical storage media. They may betransferred via networks, such as radio networks, satellite networks,wireless networks or wireline networks, e.g. the Internet.

Typical devices making use of the methods and systems described in thepresent document are portable electronic devices or other consumerequipment which are used to store and/or render audio signals.

Embodiments of the present invention may thus relate to one or more ofthe enumerated examples (EEs) listed below.

-   EEE 1. A method (600) for generating a bitstream (500) indicative of    an object based audio program, wherein the bitstream (500) comprises    a sequence of containers (501) for a corresponding sequence of audio    program frames of the object based audio program; wherein a first    container (501) of the sequence of containers (501) comprises a    plurality of substream entities (520) for a plurality of substreams    (411, 412, 413, 414) of the object based audio program; wherein the    first container (501) further comprises a presentation section    (510); wherein the method (600) comprises    -   determining (601) a set of object channels (424) indicative of        audio content of at least some of a set of audio signals;        wherein the set of object channels (424) comprises a sequence of        sets of object channel frames;    -   providing (602) a set of object related metadata (434, 444) for        the set of object channels (424); wherein the set of object        related metadata (434, 444) comprises a sequence of sets of        object related metadata frames; wherein a first audio program        frame of the object based audio program comprises a first set of        object channel frames and a corresponding first set of object        related metadata frames;    -   inserting (603) the first set of object channel frames and the        first set of object related metadata frames into a respective        set of object channel substream entities (520) of the plurality        of substream entities (520) of the first container (501); and    -   inserting (604) presentation data into the presentation section        (510); wherein the presentation data is indicative of at least        one presentation (401); wherein a presentation (401) comprises a        set of substream entities (520) from the plurality of substream        entities (520) which are to be presented simultaneously.-   EEE 2. The method (600) of EEE 1, wherein a presentation (401)    comprises one or more object channel substream entities (520) from    the set of object channel substream entities.-   EEE 3. The method (600) of any previous EEE, wherein the    presentation data is indicative of a plurality of presentations    (401, 402, 403) comprising different sets of substream entities    (520); wherein the different sets of substream entities (520)    comprise different combinations of the set of object channel    substream entities (520).-   EEE 4. The method (600) of any previous EEE, wherein the    presentation data is segmented into different presentation data    entities (512) for different presentations (401, 402, 403).-   EEE 5. The method (600) of EEE 4, further comprising, inserting    table of content data, referred to as TOC data, into the    presentation section (510); wherein the TOC data is indicative of    -   a position of the different presentation data entities (512)        within the presentation section (510); and/or    -   an identifier for the different presentations (401, 402, 403)        comprised within the presentation section (510).-   EEE 6. The method (600) of any previous EEE, wherein a substream    entity (520) comprises a content sub-entity (521) for audio content    (424) and a metadata sub-entity (522) for related metadata (434,    444).-   EEE 7. The method (600) of any previous EEE, wherein    -   a metadata frame for a corresponding channel frame comprises a        plurality of different variants (434, 444) of metadata; and    -   a presentation (401) is indicative of which variant (434) of        metadata is to be used for rendering the corresponding channel        frame.-   EEE 8. The method (600) of any previous EEE, further comprising,    -   determining a bed of speaker channels (421) indicative of audio        content of one or more of the set of audio signals; wherein the        bed of speaker channels (421) comprises a sequence of speaker        channel frames; wherein the first audio program frame of the        object based audio program comprises a first speaker channel        frame of the bed of speaker channels (421); and    -   inserting the first speaker channel frame into a speaker channel        substream entity (520) of the plurality of substreams entities        (520) of the first container (501).-   EEE 9. The method (600) of EEE 8, wherein a presentation (401) also    comprises the speaker channel substream entity (520).-   EEE 10. The method (600) of any of EEEs 8 to 9, wherein the bed of    speaker channels (421) comprises one or more speaker channels to be    presented by one or more speakers (36), respectively, of a    presentation environment.-   EEE 11. The method (600) of any of EEEs 8 to 10, wherein    -   the method (600) further comprises providing speaker related        metadata (431, 441) for the bed of speaker channels (421);    -   the speaker related metadata (431, 441) comprises a sequence of        speaker related metadata frames; and    -   a first speaker related metadata frame from the sequence of        speaker related metadata frames is inserted into the speaker        channel substream entity (520).-   EEE 12. The method (600) of any of EEEs 8 to 11, wherein the bed of    speaker channels (421) comprises one or more of: 2.0 channels, 5.1    channels, and/or 7.1 channels.-   EEE 13. The method (600) of any previous EEE, wherein the set of    object channels (424) comprises a plurality of object channels    (424).-   EEE 14. The method (600) of any previous EEE, wherein an object    channel (424) is to be presented by a combination of speakers (36)    of a presentation environment.-   EEE 15. The method (600) of EEE 14, wherein the object related    metadata (434, 444) of an object channel (424) is indicative of a    position within the presentation environment from which the object    channel (424) is to be rendered.-   EEE 16. The method (600) of any of EEEs 14 to 15, wherein    -   the position of the object channel (424) is time-varying;    -   a combination of speakers (36) for rendering the object channel        (424) changes along the sequence of object channel frames of the        object channel (424); and/or    -   a panning of the speakers (36) of the combination of speakers        (36) changes along the sequence of object channel frames of the        object channel (424).-   EEE 17. The method (600) of any previous EEE, wherein the bitstream    (500) is an AC-4 bitstream.-   EEE 18. The method (600) of any previous EEE, wherein the set of    audio signals is indicative of captured audio content.-   EEE 19. The method (600) of any previous EEE, wherein    -   a presentation (401) comprises target device configuration data        for a target device configuration; and    -   the target device configuration data is indicative of a set of        substream entities (520) from the plurality of substream        entities (520) and/or of a variant (434) of metadata, which are        to be used for rendering the presentation (401) on the target        device configuration.-   EEE 20. The method (600) of any previous EEE, wherein    -   one, two or more of the plurality of substreams form a substream        group; and    -   a presentation (401) is indicative of the substream group.-   EEE 21. The method (600) of EEE 20, further comprising inserting    data for identifying the one, two or more of the plurality of    substreams into a substream group entity of the presentation section    (510); wherein the substream group entity comprises data for    defining the substream group.-   EEE 22. A bitstream (500) indicative of an object based audio    program, wherein    -   the bitstream (500) comprises a sequence of containers (501) for        a corresponding sequence of audio program frames of the object        based audio program;    -   a first container (501) of the sequence of containers (501)        comprises a first audio program frame of the object based audio        program;    -   the first audio program frame comprises a first set of object        channel frames and a corresponding first set of object related        metadata frames;    -   the first set of object channel frames is indicative of audio        content of at least some of a set of audio signals;    -   the first container (501) comprises a plurality of substream        entities (520) for a plurality of substreams (411, 412, 413,        414) of the object based audio program;    -   the plurality of substream entities (520) comprises a set of        object channel substream entities (520) for the first set of        object channel frames, respectively;    -   the first container (501) further comprises a presentation        section (510) with presentation data;    -   the presentation data is indicative of at least one presentation        (401) of the object based audio program; and    -   a presentation (401) comprises a set of substream entities (520)        from the plurality of substream entities (520) which are to be        presented simultaneously.-   EEE 23. The bitstream (500) of EEE 22, wherein    -   the first audio program frame comprises a first speaker channel        frame of a bed of speaker channels (421);    -   the bed of speaker channels (421) is indicative of audio content        of one or more of the set of audio signals; and    -   the plurality of substream entities (520) comprises a speaker        channel substream entity (520) for the first speaker channel        frame.-   EEE 24. A method for generating a personalized audio program from a    bitstream (500) comprising an object based audio program; wherein    -   the bitstream (500) comprises a sequence of containers (501) for        a corresponding sequence of audio program frames of the object        based audio program;    -   a first container (501) of the sequence of containers (501)        comprises a first audio program frame of the object based audio        program;    -   the first audio program frame comprises a first set of object        channel frames of a set of object channels (424) and a        corresponding first set of object related metadata frames;    -   the set of object channels (424) is indicative of audio content        of at least some of a set of audio signals;    -   the first container (501) comprises a plurality of substream        entities (520) for a plurality of substreams (411, 412, 413,        414) of the object based audio program;    -   the plurality of substream entities (520) comprises a set of        object channel substream entities (520) for the first set of        object channel frames, respectively; and    -   the first container (501) further comprises a presentation        section (510);    -   wherein the method comprises    -   extracting presentation data from the presentation section        (510); wherein the presentation data is indicative of a        presentation (401) for the personalized audio program; wherein        the presentation (401) comprises a set of substream entities        (520) from the plurality of substream entities (520) which are        to be presented simultaneously; and    -   based on the presentation data, extracting one or more object        channel frames and corresponding one or more object related        metadata frames from the set of object channel substream        entities (520) of the first container (501).-   EEE 25. The method of EEE 24, wherein    -   the first audio program frame comprises a first speaker channel        frame of a bed of speaker channels (421);    -   the bed of speaker channels (421) is indicative of audio content        of one or more of the set of audio signals;    -   the plurality of substream entities (520) comprises a speaker        channel substream entity (520) for the first speaker channel        frame; and    -   the method further comprises, based on the presentation data,        extracting the first speaker channel frame from the speaker        channel substream entity (520) of the first container (501).-   EEE 26. A system (3) for generating a bitstream (500) indicative of    an object based audio program, wherein the bitstream (500) comprises    a sequence of containers (501) for a corresponding sequence of audio    program frames of the object based audio program; wherein a first    container (501) of the sequence of containers (501) comprises a    plurality of substream entities (520) for a plurality of substreams    (411, 412, 413, 414) of the object based audio program; wherein the    first container (501) further comprises a presentation section    (510); wherein the system (3) is configured to    -   determine a set of object channels (424) indicative of audio        content of at least some of a set of audio signals; wherein the        set of object channels (424) comprises a sequence of sets of        object channel frames;    -   determine a set of object related metadata (434, 444) for the        set of object channels (424); wherein the set of object related        metadata (434, 444) comprises a sequence of sets of object        related metadata frames; wherein a first audio program frame of        the object based audio program comprises a first set of object        channel frames and a corresponding first set of object related        metadata frames;    -   insert the first set of object channel frames and the first set        of object related metadata frames into a respective set of        object channel substream entities (520) of the plurality of        substream entities (520) of the first container (501); and    -   insert presentation data into the presentation section (510);        wherein the presentation data is indicative of at least one        presentation (401); wherein a presentation (401) comprises a set        of substream entities (520) from the plurality of substream        entities (520) which are to be presented simultaneously.-   EEE 27. A system (7) for generating a personalized audio program    from a bitstream (500) comprising an object based audio program;    wherein    -   the bitstream (500) comprises a sequence of containers (501) for        a corresponding sequence of audio program frames of the object        based audio program;    -   a first container (501) of the sequence of containers (501)        comprises a first audio program frame of the object based audio        program;    -   the first audio program frame comprises a first set of object        channel frames of a set of object channels (424) and a        corresponding first set of object related metadata frames;    -   the set of object channels (424) is indicative of audio content        of at least some of a set of audio signals;    -   the first container (501) comprises a plurality of substream        entities (520) for a plurality of substreams (411, 412, 413,        414) of the object based audio program;    -   the plurality of substream entities (520) comprises a set of        object channel substream entities (520) for the first set of        object channel frames, respectively; and    -   the first container (501) further comprises a presentation        section (510);    -   wherein the system (7) is configured to    -   extract presentation data from the presentation section (510);        wherein the presentation data is indicative of a presentation        (401) for the personalized audio program; wherein the        presentation (401) comprises a set of substream entities (520)        from the plurality of substream entities (520) which are to be        presented simultaneously; and    -   based on the presentation data, extract one or more object        channel frames and corresponding one or more object related        metadata frames from the set of object channel substream        entities (520) of the first container (501).

1. A method for rendering an audio program from an encoded bitstream,the method comprising: extracting presentation data from the encodedbitstream, wherein the presentation data is indicative of a presentationof the audio program, and wherein the presentation data corresponds to aset of substream entities; and rendering, based on the presentationdata, a first set of object channel frames and metadata corresponding tothe set of substream entities, wherein the audio program comprises aplurality of substreams that includes the set of substream entities, andwherein the audio program comprises a first audio program frame, whereinthe first audio program frame is part of a set of object channels and acorresponding first set of object related metadata frames; wherein anobject channel frame is to be presented by a combination of speakers ofa presentation environment, wherein an object related metadata frame ofan object channel frame is indicative of a position within thepresentation environment from which the object channel frame is to berendered; and wherein the set of object channels is indicative of audiocontent of at least some of a set of audio signals.
 2. The method ofclaim 1, wherein the first audio program frame comprises a first speakerchannel frame of a bed of speaker channels; wherein the bed of speakerchannels is indicative of audio content of one or more of the set ofaudio signals; wherein the plurality of substream entities comprises aspeaker channel substream entity for the first speaker channel frame;and wherein the method further comprises, based on the presentationdata, extracting the first speaker channel frame from the speakerchannel substream entity of the first container.
 3. A system forrendering an audio program from an encoded bitstream, the systemcomprising: a processor for extracting presentation data from theencoded bitstream, wherein the presentation data is indicative of apresentation of the audio program, and wherein the presentation datacorresponds to a set of substream entities; and a renderer forrendering, based on the presentation data, a first set of object channelframes and metadata corresponding to the set of substream entities,wherein the audio program comprises a plurality of substreams thatincludes the set of substream entities, and wherein the audio programcomprises a first audio program frame, wherein the first audio programframe is part of a set of object channels and a corresponding first setof object related metadata frames; wherein an object channel frame is tobe presented by a combination of speakers of a presentation environment,wherein an object related metadata frame of an object channel frame isindicative of a position within the presentation environment from whichthe object channel frame is to be rendered; and wherein the set ofobject channels is indicative of audio content of at least some of a setof audio signals.
 4. A non-transitory storage medium having a bitstreamconfigured to render according to the method of claim 1.